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Server identifies a document (or documents) accessed or 
viewed by a user using the user system 
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Server determines which sections of the document 
identified in step 302 are to be analyzed 
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Server extracts (or generates) UDIOs from the sections of 
the document determined in step 304 
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UDIOs extracted in step 306 are analyzed 
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Server selects CPIOs to be provided to the user 
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CPIOs selected in step 310 are communicated to the user 

system 
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User system outputs information contained in the selected 

CPIOs 
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Fig. 3 
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From step 306 in Fig. 3 
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Determine the "type" or "class" of each extracted UDIO 
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For each extracted UDIO, based upon the class of the 
UDIO (as detemnined in step 402), detemnine one or more 
content recognition techniques to be applied to the UDIO 
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For each extracted UDIO, determine the contents of the 
UDIO by applying the content recognition techniques for 
the UDIO detemnined in step 404 
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Store the UDIOs. their associated class information, and 
their content information 
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To step 310 in Fig. 3 
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From step 308 in Fig. 3 



Identify one or more selection techniques to be used for 
selecting CPIOs to be provided to the user 
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Apply each selection technique identified in step 502, to 
determine relevancy scores between the CPIOs and the 
UDIOs for each selection technique 
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Combine the results of the various selection techniques 

applied in step 504 to generate a single "final" list of 
selected CPIOs to be communicated to the user system 
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To step 312 in Fig. 3 
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From step 308 in Fig. 3 
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Determine concepts of interest to 
a user 






For each UDIO, determine a 
degree of relevancy of the UDIO 
to each concept of interest to the 

user determined in step 602 







Based upon the relevancy of the 
UDIOs and CPIOs to the user 
concepts of interest, for each 

CPIO. detennine the degree of 
relevancy of the CPIO to each 

UDIO as it relates to a particular 
user concept of interest 
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Determine one or more 
comparison techniques to be 
applied to each UDIO based upon 
the "class" of the UDIO 
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For each UDIO, each comparison 
technique associated with UDIO is 

applied to determine relevancy 
between the UDIO and the CPIOs 



Generate a single "final" list of 
CPIOs using a combination 
technique 



Select one or more CPIOs based 

upon the relevancy scores 
detennined for the CPIOs in step 
606 
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Select one or more CPIOs based 

upon the relevancy scores 
associated with the CPIOs in step 
612 
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Parameters: 
Weighting Factors: (W1, W2. . . .)- 
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Source #1: List of CPIOs 



and relevancy info 
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and relevancy info 



Source #1: List of CPIOs 



and relevancy info 



:T1,T2....) 



706-a 



Combination 
Technique 



A single "final" 
list of CPIOs ~^ 
708 



702 



Fig. 7 
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Initialize variables: 
Array NumVotes[1..N] = 0 
Array WeightVotes[1 ..N] = 0 
Array of Lists SourcesVoting[1 ..N] = Null 
#S = Total number of sources 
d = 1 
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Select source 
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#L = Total number of ListElements in the list for source 

e=1 
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CPIOjg = CPIO object in ListElementg of source 



= Relevance value in ListElements of source for CPIO, 
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NumVotes[CPIOJ = NumVotes[CPIOJ + 1 
WeightVotes[CPIO J = WeightVotes[CPIO J + Wg^ 
Append to list SourcesVoting[CPIO J 



e = e + 1 
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d = d + 1 



Generate a "final" list of one or CPIOs based on specified 

critieria 
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To step 312 in Fig. 3 
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